Electronically Screening Discharge Summaries for Adverse Medical Events

Design: A cohort study including 424 randomly selected admissions to the medical services of an academic medical center was conducted between January and July 2000. The authors developed a computerized screening tool that searched free-text discharge summaries for trigger words representing possible adverse events. Measurements: All discharge summaries with a trigger word present underwent chart review by two independent physician reviewers. The presence of adverse eventswas assessed using structured implicit judgment. A random sample of discharge summaries without trigger words also was reviewed. Results: Fifty-nine percent (251 of 424) of the discharge summaries contained trigger words. Based on discharge summary review, 44.8% (327 of 730) of the alerted trigger words indicated a possible adverse event. After medical record review, the tool detected 131 adverse events. The sensitivity and specificity of the screening tool were 69% and 48%, respectively. The positive predictive value of the tool was 52%. Conclusion: Medical discharge summaries contain information regarding adverse events. Electronic screening of discharge summaries for adverse events using keyword searches is feasible but thus far has poor specificity. Nonetheless, computerized clinical narrative screening methods could potentially offer researchers and quality managers a means to routinely detect adverse events. j J Am Med Inform Assoc. 2003;10:339–350. DOI 10.1197/jamia.M1201. Patient safety has emerged as a highly important issue for health care. Adverse events (AEs)—defined as injuries due to medical management—result in numerous injuries and deaths every year within the United States. Prior studies have found rates of AEs ranging from 2.9% to 16.6% of inpatient admissions. These injury rates have prompted the Institute of Medicine to define patient safety as a key goal in health care quality improvement. One of the first laws of quality improvement is that to improve something, one must be able to measure it. However, most organizations do not have effective approaches to routinely detect and measure AEs. Voluntary incident reports underestimate AE rates, detecting 1.5% of AEs and 6% of adverse drug events compared with manual chart review. While chart review is effective for research, it is too costly for routine use. Identification of AEs by searching electronic patient records, rather than manually reviewing paper charts, offers a potential solution. Studies have found that computerized screening for adverse drug events (ADEs) requires 20% of the time and detects 69% as many cases compared with Affiliations of the authors: Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, Massachusetts. Dr. Murff and Dr. Peterson are currently in the Division of General Internal Medicine, Vanderbilt University Medical Center, Nashville, TN; Ms. Fiskio, Dr. Heiman, and Dr. Bates are currently in the Division of General Internal Medicine, Brigham & Women’s Hospital, and Harvard Medical School, Boston, MA; Dr. Forster is currently at the University of Ottawa, Clinical Epidemiology Unit, Ottawa Health Research Unit, Ottawa Hospital, Ottawa, Ontario, Canada. A part of this material has been presented as a poster at the 2001 American Medical Informatics Association Annual Symposium (Murff HJ, Forster AJ, Peterson JF, Fiskio JM, Heiman HL, Bates DW. Electronically screening discharge summaries for adverse medical events. J Am Med Inform Assoc. 2002;9(6 suppl):S50–1) and the 2002 Society of General Internal Medicine National Meeting (Murff HJ, Forster AJ, Peterson JF, Fiskio JM, Heiman HL, Bates DW. Electronically screening discharge summaries for adverse medical events. J Gen Intern Med. 2002;17(suppl 1):A205). Dr. Murff was supported by a NRSA training grant, 5T321101-12, over the duration of the project. Correspondence and reprints: David W. Bates, MD, MSc, Chief, Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA 02115; e-mail: . Received for publication: 08/08/02; accepted for publication: 01/29/03. 339 Journal of the American Medical Informatics Association Volume 10 Number 4 Jul / Aug 2003